This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 



BEST AVAILABLE IMAGES 

Defective images within this document are accurate representation of 
The original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 



IMAGES ARE BEST AVAILABLE COPY. 

As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 



(oidsn) 30Vd SIH1 



CM 
< 

CO 
00 

o 



(19) 



J 




(12) 



Europaisches Patentamt 
Europ an Patent Office 
Office europeen des brevets (11) EP 1 081 619 A2 

EUROPEAN PATENT APPLICATION 

(51) IntCI 7 : G06F 17/60 



(43) Date of publication: 

07.03.2001 Bulletin 2001/10 

(21) Application number: 00303005.3 

(22) Date of filing: 10.04.2000 



(84) Designated Contracting States: 


(72) 


Inventor: 


AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 




Nagae, Takaaki, 


MC NL PT SE 




c/o Fujitsu Limited 


Designated Extension States: 




Kawasaki-shi, Kanagawa 211-8588 (JP) 


AL LT LV MK RO SI 








(74) 


Representative: 


(30) Priority: 30.08.1999 J P 24387299 




Hitching, Peter Matthew et al 




Haseltine Lake & Co., 


(71) Applicant: FUJITSU LIMITED 




Imperial House, 


Kawasaki-shi, Kanagawa 211-8588 (JP) 




15-19 Kings way 
London WC2B 6UD (GB) 



(54) Electronic mail apparatus and method 

(57) An electronic mail transmission method is pro- 
vided such that, when a sentence serving as the con- 
tents of an electronic mail written (S01 ) in Japanese (for 
example) cannot be text-displayed in a destination of 
the electronic mail (S09), a text file corresponding to the 
sentence is converted into a bit map file serving as 
image data (S 10), and the bit map file is transmitted to 
the destination of the electronic mail (S1 1 ). In this man- 
ner, even if Japanese cannot be text-displayed in the 
destination, an electronic mail written in Japanese can 
be read in the destination. 
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D scription 

[0001] The present invention relates to an elec- 
tronic mail apparatus and method. 
[0002] In recent years, use of electronic mail has 
spread rapidly on a world-wide scale. In addition, in 
accordance with global development of enterprises, the 
importance of electronic mails increases. Persons hav- 
ing friends residing in foreign countries increase in 
number, and persons having different mother tongues 
are frequently in correspondence with each other with 

lectronic mails. 
[0003] An electronic mail can be transmitted to any- 
one in the world in an environment which can be con- 
nected to a network. For this reason, if the difficulties of 
communication caused by differences between lan- 
guages can be overcome, it is supposed that the elec- 
tronic mail can become a convenient tool which can be 
easily used by anyone. 

[0004] However, Japanese is expressed by hira- 
gana, katakana, and kanji , where the number of types 
of characters to be expressed is several thousands to 
several tens of thousands. For this reason, in Japan, a 
2-byte character code is used to express one character. 
[0005] In contrast to this, English is expressed by 
only roman letters (alphabets). The number of types of 
alphabets is fifty-two. For this reason, in a country (e.g., 
in the U.S.A.) using English as its mother tongue, a 1- 
byte character code (8-byte code) is used to express 
one character (e.g., the ASCII code). 
[0006] In contrast to this, German is expressed by 
roman letters and characters obtained by adding 
umlauts to the roman letters. For this reason, the 
number of types of used characters is larger than Eng- 
lish. Therefore, in Germany, in order to express one 
character, a 1.5-byte character code is used. 
[0007] As described above, since the types of char- 
acters used are different in number depending on the 
language. For this reason, in the world, the byte counts 
of several types of character codes are used. 
[0008] A character display device (e.g., a personal 
computer, a work station, a mobile computer, a mobile 
telephone set, or the like) for displaying text characters 
using character codes is operable to handle character 
codes (text data) which can express the language of the 
country in which the device is used. 
[0009] However, since character codes which are 
handled by the character display device are different 
from each other, the following problem occurs. In gen- 
eral, character codes which can be handled by the char- 
acter display device are limited to character codes 
which can display text of the language of the country in 
which the device is used. 

[001 0] For this reason, the character display device 
cannot display text characters by using character codes 
other than the character codes which can be handled by 
the device. For this reason, since a character display 
device used in the U.S.A. can handle only 1 -byte char- 



acter codes, the character display device cannot handle 
1 .5-byte and 2-byte character codes. More specifically, 
Japanese and German text cannot be displayed. 
[0011] In addition, some character codes having 
5 equal byte counts have difference character codes 
which can be handled (Japanese, Korean, and the like). 
In such a case, the same problem as described above 
occurs. 

[0012] Therefore, for example, supposing that a 

10 Japanese person creates an electronic mail in Japa- 
nese to transmit to her/his friend residing in the U.S.A. 
the character display device used by the friend can 
receive only 1-byte codes, and consequently the char- 
acter display device cannot display the text contents of 

15 the electronic mail. 

[0013] In this manner, character codes (text data) 
corresponding to the contents of the electronic mail can 
be transmitted and received around the world. However, 
when the character codes cannot be handled on the 

20 reception side, the text contents of the electronic mail 
cannot be displayed on the reception side. 
[0014] It is therefore desirable to provide an elec- 
tronic mail apparatus which can read an electronic mail 
on a reception side formed in a language used on a 

25 transmission side even if characters expressed in the 
language used in the transmission side cannot be dis- 
played on the reception side. 

[0015] According to an embodiment of the present 
invention there is provided an electronic mail apparatus 

30 comprising: an image data forming unit using a sen- 
tence input as the contents of an electronic mail to form 
image data corresponding to the sentence; and a trans- 
mission unit transmitting the formed image data to a 
destination of the electronic mail. 

35 [0016] In an electronic mail system, most comput- 
ers used as terminal devices can handle image files 
(e.g., bit map files), and can display images based on 
the image files. More specifically, the image files can be 
handled in common in most computers around the 

40 whole world. 

[0017] In an embodiment of the present invention, 
the image data of a sentence constituting the contents 
of the electronic mail are transmitted. For this reason, 
for example, even when characters expressed in a lan- 

45 guage used by the transmission source cannot be text- 
displayed on the reception side, an image based on the 
image data can usually be displayed. Therefore, on the 
reception side, a sentence of the electronic mail based 
on the image data can usually be displayed. The 

so receiver of the electronic mail can read the text of the 
electronic mail from the image of the sentence using 
characters expressed in the language used by the 
transmission source. 

[0018] An embodiment of the present invention 
55 preferably further comprises a check unit checking 
whether a sentence should be transmitted as text data 
or image data, and the image data forming unit may 
form image data corresponding to the sentence by 
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using the sentence if it is determined by the check unit 
that the image data should be transmitted. 
[0019] In addition, in an embodiment of the present 
invention, the check unit preferably determines that the 
sentence should be transmitted as image data if it is 5 
determined that the text data of the sentence is not cor- 
rectly displayed. 

[0020] An embodiment of the present invention 
preferably further comprises a display control unit dis- 
playing a sentence on a screen by using the text data of 10 
a sentence, and the image data forming unit may form 
image data corresponding to the sentence in a format of 
the sentence displayed on the screen by the display 
control unit. 

[0021] An embodiment of the present invention 15 
preferably further comprises a translation unit translat- 
ing, when a sentence is expressed in a language differ- 
ent from a language used in a destination of the 
electronic mail, the sentence into the language used in 
the destination. The transmission unit may transmit 20 
translation results obtained by the translation unit to the 
destination of the electronic mail. 
[0022] In an embodiment of the present invention, 
the image data forming unit preferably forms the trans- 
lation image data of the translation results expressed by 25 
characters written in the language used in the destina- 
tion when the display control unit cannot text-display 
characters written in the language used in the destina- 
tion of the electronic mail, and the display control unit 
may display the translation image data on a screen such 30 
that the translation image data can be edited. 
[0023] In an embodiment of the present invention, 
when the display control unit cannot text-display charac- 
ters written in the language used in the destination of 
the electronic mail, the image data forming unit prefera- 35 
bly forms the translation image data of the translation 
results expressed by characters written in the language 
used in the destination. Such an embodiment of the 
present invention further comprises a synthesizing unit 
forming synthesized image data obtained by combining 40 
the image data corresponding to the input sentence 
formed by the image data forming unit to the translation 
image data, and the transmission unit may transmit the 
synthesized image data to the destination of the elec- 
tronic mail. 45 
[0024] In an embodiment of the present invention, 
the transmission unit preferably transmits the image 
data as an attached file to the destination of the elec- 
tronic mail. 

[0025] According to an embodiment of the present so 
invention, even if characters written in the language 
used on the transmission side cannot be text-displayed 
on the reception side, an electronic mail formed in the 
language used on the transmission side can be read on 
the reception side. 55 
[0026] Reference will now be made, by way of 
example, to the accompanying drawings, in which: 
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Fig. 1 is a diagram for explaining an electronic mail 
system according to an embodiment of the present 
invention. 

Rg. 2 is a block diagram showing the hardware of 

the terminal devices shown in Fig. 1 . 

Rg. 3 is a flow chart showing an operation in the 

terminal device shown in Fig. 2. 

Fig. 4 is a functional block diagram of the terminal 

device shown in Fig. 2. 

Rg. 5 is a diagram showing a display example of an 

edition screen obtained by a text editor. 

Fig. 6 is a diagram showing a display example of an 

edition region shown in Fig. 5. 

Rg. 7 is a table for explaining the address list 

shown in Fig. 4. 

Rg. 8 is a diagram for explaining a process per- 
formed by the translation unit shown in Fig. 4. 
Rg. 9 is a diagram for explaining a process per- 
formed by the translation unit shown in Fig. 4. 
Rg. 1 0 is a diagram for explaining image file forma- 
tion performed by the image forming unit shown in 
Rg- 4. 

Rg. 11 is a diagram for explaining an edition 
screen. 

Rg. 12 is a diagram for explaining an edition 
screen. 

Rg. 13 is a diagram for explaining display control 
performed by the display control unit shown in Rg. 
4. 

Rg. 14 is a diagram for explaining an editing opera- 
tion. 

Rg. 15 is a diagram for explaining a synthesizing 
process performed by a synthesizing unit. 
Rg. 16 is a diagram for explaining a process per- 
formed by a transmission unit. , 
Fig. 17 is a flow chart showing an operation per- 
formed by a mail server. 

Rg. 18 is a functional block diagram of a mail 
server. 

[0027] An embodiment of the present invention will 
be described below with reference to the drawings. 

(Electronic Mail System) 

[0028] Fig. 1 is a diagram for explaining an elec- 
tronic mail system according to an embodiment of the 
present invention. In Fig. 1, the electronic mail system is 
constituted by a plurality of terminal devices connected 
to a network. In Fig. 1, for example, an electronic mail 
system in which a plurality of terminal devices T1, T2, 
and T3 are connected to each other through a network 
NW is shown. 

[0029] The terminal devices T1 , T2, and T3 are, for 
example, personal computers, work stations, mobile 
computers (mobile terminals), or mobile telephone sets. 
[0030] The terminal device T1 is arranged in Japan, 
and can text-display Japanese and English by using 2- 
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byte character codes. In addition, the terminal device T2 
is arranged in the U.S.A., and can text-display only Eng- 
lish by using 1-byte character code. Furthermore, the 
terminal device T3 is arranged in Germany, and can 
text-display English and German by using 1.5-byte 
character codes. 

[0031] Rg. 2 is a block diagram showing example 
hardware of the terminal devices T1 , T2, and T3 shown 
in Fig. 1 . Since the terminal devices T1 , T2, and T3 have 
the same hardware configurations, the terminal device 
T1 will be described for example. 

[0032] In Fig. 2, the terminal device T1 comprises a 
CPU (Central Processing Unit) 16, a ROM (Read Only 
Memory) 17, a RAM (Random Access Memory) 18, a 
hard disk drive (HDD) 19, a floppy disk drive (FDD) 20, 
a CD-ROM driver 21 , a graphic board 22, a communica- 
tion control device 23, and interface circuits (l/Fs) 24 
and 25 which are connected to each other through a 
bus BUS. 

[0033] The graphic board 22 is connected to a dis- 
play device 41 such as a cathode ray tube (CRT) or a 
liquid crystal display (LCD). The l/F 24 is connected to a 
keyboard (KBD) 27. The l/F 25 is connected to a mouse 
28 or a pointing device such as a track ball, a flat space, 
or a joy stick. 

[0034] The ROM 17 stores a booting program 
therein. The booting program is executed by the CPU 
16 when the terminal device T1 is powered on. In this 
manner, an operating system (OS) stored in the HDD 1 9 
and a single driver or a plurality of drivers for a display 
process or a communication process are loaded on the 
RAM 18 to make it possible to execute various proc- 
esses and controls. 

[0035] In the RAM 18, a program for controlling the 
terminal device T1 is developed. In addition, the RAM 
18 holds a process result obtained by a program, tem- 
porary data for a process, displaying data for displaying 
the process result or the like on the screen of the display 
26, and the like. The RAM 1 8 is used as a work region 
for the CPU 16. 

[0036] The displaying data developed on the RAM 
1 8 is transmitted to the display 26 through the graphic 
board 22. The display 26 displays display contents cor- 
responding to the displaying data on the screen of the 
display 26. 

[0037] The HDD 19 is a device for recording or 
reading a program, control data, text data, image data, 
or the like on/from a hard disk in accordance with an 
instruction from the CPU 16. 

[0038] The FDD 20 is a device for recording or 
reading a program, control data, text data, image data, 
or the like on/from a floppy disk 29 in accordance with 
an instruction from the CPU 1 6. 

[0039] The CD-ROM driver 21 is a device for read- 
ing a program or data recorded on a CD-ROM (read 
only memory using a compact disk) 30 in accordance 
with an instruction from the CPU 16. 
[0040] The communication control device 23 exe- 



cutes transmission/reception of data to/from another 
device or downloading of a program or data by using a 
communication line connected to the terminal device T1 
in accordance with an instruction from the CPU 16. 

5 [0041] The KBD 27 comprises a plurality of keys 
(character input keys, cursor keys, and the like). The 
KBD 27 is used to cause an operator to input data in the 
terminal device T1. The mouse 28 is used to input a 
selection instruction using a mouse cursor displayed on 

io the display 26. 

[0042] The CPU 16 executes various types of pro- 
grams stored in the ROM 1 7, the hard disk, the FD 29, 
and the CD-ROM 30 (they correspond to recording 
media). In this manner, the terminal device T1 functions 

75 as an electronic mail apparatus embodying the present 
invention. The program and data held in the recording 
medium such as a hard disk may be manually recorded 
on the recording medium by the operator, and a pro- 
gram or data may be downloaded from another device 

20 to be recorded on the recording medium. 

(Process in Terminal Device T1) 

[0043] The terminal device T1 functions as an elec- 
ts tronic mail apparatus for performing the following oper- 
ation such that the CPU 16 executes a program. Fig. 3 
is a flow chart showing an operation in the terminal 
device T1 . 

[0044] First, an environment for forming an elec- 
30 tronic mail in a mother tongue is provided to a user of 
the terminal device T1 (step S01). More specifically, an 
environment for forming a sentence constituting the 
contents of an electronic mail is provided such that the 
CPU 1 6 executes an application program of a text edi- 
35 tor. 

[0045] The user operates the KBD 27 in the pro- 
vided environment to form the sentence of an electronic 
mail. At this time, the CPU 1 6 forms a text file corre- 
sponding to the formed sentence. The CPU 16 displays 
40 the sentence based on the formed text file on the screen 
of the display 26 in accordance with a predetermined 
format. 

[0046] An environment for specifying a destination 
of an electronic mail is provided to the user (step S02). 

45 More specifically, the CPU 1 6 reads address list 52 (see 
Fig. 7) of electronic mails recorded on the hard disk or 
the FD 29 in accordance with an instruction input from 
the user through the KBD 27 or the mouse 28, and the 
address list 52 is displayed on the screen of the display 

so 26. 

[0047] The user refers to the address list 52 selects 
the address corresponding to the destination by using 
the KBD 27 or the mouse 28. At this reason, when the 
address corresponding to the destination is not in the 
55 address list 52, the user inputs the corresponding 
address to the terminal device T1 by using the KBD 27. 
[0048] It is checked whether a language identical to 
the language in the transmission source of the elec- 
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tronic mall is used in the destination or not (step S03). 
At this time, if it is determined that the languages is 
identical with each other, the operation shifts to step 
S1 1 . If it is determined that the languages are different 
from each other, the operation shifts to step S04. 
[0049] The CPU 16 performs determination by 
using determination data prepared in advance. In place 
of this determination, a question whether the language 
(Japanese) used in the transmission source is identical 
to the language used in the destination may be provided 
to the user through the display 26, and the operation 
may shift to step S11 or step S04 depending on the 
answer, i.e., -identical" or "different- of the user. Also, 
the CPU 16 may analyze a language used in the desti- 
nation on the basis of the address specified in step S02 
to automatically checks whether the identical languages 
are used in the transmission source and in the destina- 
tion. 

[0050] In step S04, a language (mother tongue) 
used in the destination is specified, the sentence 
formed in step S01 is translated into the specific lan- 
guage. The language may be specified such that the 
user designates the language, and the CPU 16 may 
analyze the language on the basis of the address. 
[0051] In step S05, it is checked whether characters 
expressed in the language used in the destination can 
be text- displayed by the terminal device T1 or not. The 
CPU 16 performs the determination by using the deter- 
mination data. If it is determined that the characters can 
be text-displayed, the CPU 16 causes the operation to 
shift to step S06. If it is determined that the characters 
cannot be text-displayed, the CPU 16 causes the oper- 
ation to shift to step S07. 

[0052] In place of this, in step SOS, the CPU 1 6 may 
provide a question whether the characters expressed in 
the language in the destination can be text-displayed or 
not to the user through the display device 26 to cause 
the operation to shift to step S06 or S07 depending on 
an answer, i.e., "it is possible to perform text display" or 
"it is impossible to perform text display" from the user. 
The CPU 1 6 may automatically check whether the char- 
acters expressed in the language in the destination can 
be text-displayed or not on the basis of the address of 
the destination and character codes which can be han- 
dled by the terminal device T1 . 

[0053] In step S06, the CPU 1 6 forms the text file of 
results (translation results) of the translation process in 
step S04 in accordance with the determination result in 
step SOS, and then causes the operation to shift to step 
S08. 

[0054] In step S07, the CPU 1 6 forms the image file 
(bitmap file, JPEG file, GIF file, or the like) of the trans- 
lation results in step S04 in accordance with the deter- 
mination result in step S05. 

[0055] In step S08, the CPU 16 displays an edition 
screen for an electronic mail on the display device 26. 
On the edition screen, characters based on the text file 
of the sentence formed in step S01 and characters 



(translation results) based on the text file or the image 
file formed in step S06 or step S07 are displayed. The 
user refers to the contents displayed on the edition dis- 
play to perform an editing operation such as correction 
5 of mistranslations by using the KBD27orthe mouse 28. 
[0056] In step S09, it is checked whether characters 
expressed in the language (Japanese) used in the 
transmission source can be text-displayed in the desti- 
nation or not. If it is determined that the characters can 
10 be text-displayed, the operation shifts to step Si 1 ; oth- 
erwise, the operation shifts to step S10. 
[0057] This determination in CPU 16 is performed 
by referring to determination data which is preferred in 
advance. In place of the determination, the following 
15 operation may be performed. That is, the CPU 16 pro- 
vides a question whether the characters expressed in 
the language in the transmission source can be text-dis- 
played or not to the user through the display device 26. 
When the user inputs the answer, i.e., "it is possible to 
20 perform text display", the operation shifts to step S1 1; 
when the user inputs the answer, i.e., "it is impossible to 
perform text display", the operation shifts to step S10. 
[0058] In step S1 0, the CPU 1 6 forms an image file 
corresponding to a text file (sentence formed by channe- 
ls ters expressed in the language in the transmission 
source) formed in step S01, and a synthesized image 
file obtained by synthesizing the image file with the 
image file formed in step S07 is formed. 
[0059] In step S11, the electronic mail formed in 
30 steps S01 to S1 0 is transmitted to an address (terminal 
device T2 and/or terminal device T3) specified in step 
S02. The contents of the specified electronic mail are as 
follows. 

35 (1) When "YES" is determined in step S03, a sen- 
tence (Japanese text file) formed in a language in a 
transmission source is used as the body of an elec- 
tronic mail. 

(2) When "NO* is determined in step S03, and 
40 when "YES" is determined in step S06 and step 
S09, a synthesized text file obtained by synthesiz- 
ing a Japanese text file and the text file of transla- 
tion results is used as the body of an electronic 
mail. 

45 (3) When "NO" is determined in step S03 and step 
SOS, and when "YES" is determined in step S09, 
the synthesized file obtained by synthesizing the 
Japanese text file and the image file of the transla- 
tion results is used as an attached file. 

so (4) When "NO" is determined in steps S03 and SOS 
and step S09, the synthesized file obtained by syn- 
thesizing the image file of the translation results 
and a Japanese image file is used as an attached 
file. 

55 

{Detailed Operation of Terminal Device T1> 

[0060] Fig. 4 is a functional block diagram of a ter- 



5 



BNSDCCID: <EP 1081619A2_I_> 



9 



EP 1 081 619 A2 



10 



minal device T1 . When the CPU 16 executes a program 
to execute the operations in steps S01 to S1 1 , the termi- 
nal device T1 functions as a device comprising the dis- 
play device 41, a display control unit 42, an image 
forming unit (corresponding to an image data forming 
unit) 43, a font set (data set of font data) 44, a synthesiz- 
ing unit 45, a transmission unit 46, a first determination 
unit 47, a translation unit 48, a dictionary 49, a second 
determination unit 50, a third determination unit 51 , and 
the address list 52. 

[0061] Steps S01 to S11 shown in Fig. 3 will be 
described below by using Fig. 4 to Fig. 16. 

((Step S01» 

[0062] In step S01 , the display control unit 42 dis- 
plays the screen (edition screen) of a text editor which 
copes with n languages used in the electronic mail 
apparatus on the display device 26. 
[0063] Fig. 5 is a diagram showing a display exam- 
ple of the edition screen obtained by the text editor. In 
Fig. 5, the edition screen has a rectangular edition 
region 55, a translation button 56 arranged on the right 
side of the edition region 55, an address list button 57, 
and a transmission button 58. 

[0064] A user designates the position of a cursor 
(not shown) displayed on the address list button 57 with 
the KBD 27 or the mouse 28 to input a Japanese sen- 
tence constituting the contents of an electronic mail 
from the KBD 27. 

[0065] At this time, the display device 41 forms text 
data corresponding to the input characters. More specif- 
ically, the display device 41 has a data set (font set) of 
fonts corresponding to the JIS (Japan Industrial Stand- 
ards) code. Font data corresponding to the character 
code of the characters input from the KBD 27 is 
extracted from the font set to form text data. 
[0066] The display control unit 42 displays the Jap- 
anese characters corresponding to the formed text data 
in the edition region 55 in accordance with the informa- 
tion of a predetermined format (document style). In this 
manner, the sentence constituting the contents of the 
electronic mail is formed in Japanese. The text file cor- 
responding to this sentence is formed by the display 
d vice 41 . 

[0067] Fig. 6 is a diagram showing a display exam- 
ple in which formed sentences are displayed in the edi- 
tion region 55. As shown in Fig. 6, the formed sentences 
are text-displayed in the edition region 55 at intervals 
each corresponding to one line. 

[0068] The display device 41 is a function realized 
such that the CPU 16 executes the program of the text 
editor, and the display control unit 42 is a function real- 
ized such that the CPU 16 controls the graphic board 22 
according to a predetermined control program. 



((Step S02» 

[0069] In step S02, a user clicks the address list 
button 57 displayed on the edition screen with the 
5 mouse 28 (or presses the address list button 57 by 
operating the KBD 27). At this time, the address list 52 
recorded on the hard disk or the FD 29 is read, and the 
contents of the address list 52 are displayed on the dis- 
play device 26. 

w [0070] Fig. 7 is a table for explaining an example of 
the address list 52. As shown in Fig. 7, the address list 
52 is constituted by a plurality of records including 
pieces information corresponding to the respective 
items, i.e., "transmission target", "address list", "mail 

15 address", "mother tongue", and "display in Japanese". 
[0071] In this example, the mail addresses of "Bob" 
residing in the U.S.A. and having the terminal device T2, 
"Mizel" residing in Germany and having the terminal 
device T3, and "Yamada" residing in Japan are regis- 

20 tered. 

[0072] The user operates the KBD 27 or the mouse 
28 to check records, so that the user can selects a 
transmission target (destination). At this time, the user 
can select a plurality of destinations. When the plurality 
25 of destinations are selected, a formed electronic mail is 
multi-casted to the destinations. In this example, it is 
assumed that "Bob" and "Mizer are selected as desti- 
nations. 

30 ((Step S03» 

[0073] In step S03, the first determination unit 47 
checks whether the language in the transmission 
source is identical to the languages in the destinations 

35 selected in step S02 or not. More specifically, Ifie; first 
determination unit 47 holds information representing : -~ 
that the language in the transmission source is Japa- 
nese to check whether the "mother tongue" in the 
record checked in step S02 is Japanese or not. 

40 [0074] As described above, the first determination 
unit 47 automatically performs determination by using 
the language (Japanese) in the transmission source 
prepared as determination data and the information of 
"mother tongue" of the address list 52. When the lan- 

45 guage in the transmission source is different from the 
language in the destination (the language in the desti- 
nation is not Japanese), the first determination unit 47 
notifies the translation unit 48 that the languages are dif- 
ferent from each other. In contrast to this, when the lan- 

so guage in the destination is Japanese, the first 
determination unit 47 does not notify the translation unit 
48. 

[0075] In this example, the first determination unit 
47 determines that the languages of "Bob" and "Mizel" 
55 are not Japanese, and the first determination unit 47 
notifies the translation unit 48 that the languages are not 
Japanese. The first determination unit 47 is a function 
realized such that the CPU 16 execut s a program. 
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{{Step S04» 

[0076] In step S04, the translation unit 48 is started 
by receiving the notification from the first determination 
unit 47 to perform the following processes. That is, the 
translation unit 48 specifies a language to be translated 
with reference to "mother tongue" in the record corre- 
sponding to the destination in which it is determined that 
the language is not Japanese. 

[0077] In this manner, the translation unit 48 speci- 
fies the language (mother tongue) of the destination 
"Bob" as English, and specifies the language (mother 
tongue) of the destination "Mizel" as German. In this 
example, although the translation unit 48 specifies a 
language to be translated, the first determination unit 47 
may specifies the language to be translated to notify the 
translation unit 48 of the language. 
[0078] Subsequently, the translation unit 48 trans- 
lates a sentence in Japanese (Japanese sentence) 
formed in step S01 into a specific language with refer- 
ence to the dictionary (basic dictionary) 49. In this case, 
the dictionary 49 is constituted by dictionaries of n lan- 
guages such as a Japanese-English dictionary or a Jap- 
anese-Germany dictionary for translating Japanese into 
a foreign language. 

[0079] The translation unit 48 extracts words or idi- 
oms of one sentence when the translation unit 48 trans- 
lates a Japanese sentence, and translates the extracted 
words and idioms into the foreign language correspond- 
ing to the mother tongue in the destination. When the 
words or the idioms are extracted, for example, decom- 
position of parts of speech is performed by explosion 
morpheme analysis, and a translation for a specific part 
of speech (e.g., noun or verb) is performed. 
[0080] The translation unit 48 allocates data repre- 
senting an impossible translation to a word which is not 
described in the dictionary 49. The translation unit 48 
may translate not only the words and idioms in one sen- 
tence but also the whole sentence in place of the proc- 
ess described above. 

[0081] Fig. 8 is a diagram for explaining a process 
performed by the translation unit 48. The translation unit 
48 has a learning function, and words and idioms 
obtained by the learning function are registered on a 
learning dictionary 48a included in the translation unit 
48. In this manner, the translation unit 48 reflects past 
translation results on the present translation process. 
[0082] In this example, the translation unit 48 trans- 
lates Japanese into English as a translation corre- 
sponding to the destination "Bob", and translates 
Japanese into Germany as a translation corresponding 
to the destination "Mizel". 

[0083] Fig. 9 is a diagram for explaining an internal 
process in which the translation unit 48 translates the 
Japanese sentence shown in Fig. 6. Fig. 9A shows a 
translation from Japanese to English performed by the 
translation unit 48, and Fig. 9B shows a translation from 
Japanese to German performed by the translation unit 



48. 

[0084] The translation unit 48 is a function realized 
such that the CPU 16 executes a translation program. 
As the translation program, any conventional translation 
5 programs may be used. 

{{Step S05)> 

[0085] In step S05, the second determination unit 
10 50 checks whether characters expressed in languages 
in destinations selected in step S02 can be displayed by 
the terminal device T1 . More specifically, the second 
determination unit 50 holds the information of a charac- 
ter code which can be handled by the terminal device 
15 T1. 

[0086] In this example, the terminal device T1 can 
handle the JIS code, and can text-display hiragana, 
katakana, kanji, roman tetters, and the like. However, 
the terminal device T1 cannot text-display characters 

20 obtained by adding umlauts to the roman letters. 

[0087] Therefore, the second determination unit 50 
determines that English translation results of the desti- 
nation "Bob- can be text-displayed, and determines that 
German translation results of the destination "Mizel" 

25 cannot be text-displayed. If it is determined that it is pos- 
sible to perform text display, the second determination 
unit 50 notifies the display device 41 that it is possible to 
perform text display. If it is determined that it is impossi- 
ble to perform text display, the second determination 

30 unit 50 notifies the image forming unit 43 that it is impos- 
sible to perform text display. 

[0088] The second determination unit 50 is a func- 
tion realized such that the CPU 16 executes a program. 

35 ((Step S06)> 

[0089] In step S06, the display device 41 receives 
translation results from the translation unit 48 in accord- 
ance with a notification from the second determination 
40 unit 50. The display device 41 forms text data corre- 
sponding to words and idioms included in the translation 
results and translated into a foreign language which can 
be text-displayed in the terminal device T1 to holds the 
text data as a text file. 

45 

{{Step S07» 

[0090] In step S07, the image forming unit 43 
receives translation results from the translation unit 48 

so in accordance with a notification from the second deter- 
mination unit 50. The image forming unit 43 forms 
image data corresponding to words and idioms included 
in the translation results and translated into a foreign 
language which can be text-displayed in the terminal 

55 device T1 by using the font set 44 to hold the image data 
as an image file. 

[0091] The font set 44 is stored in the ROM 1 7, the 
hard disk, or the FD 29, arid includes font (character 
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image) data of characters for writing a foreign language, 
which cannot be text-displayed in the terminal device 
T1 , of the n languages described above. For example, 
the font set 44 includes the font data of characters for 
writing German. 5 
[0092] Fig. 1 0 is a diagram for explaining image file 
formation performed by the image forming unit 43. Fig. 
10 shows, for example, a case in which the image files 
of words translated into Germany is formed. As shown 
in Fig. 10, the image files (bitmap files) of translated 10 
sentences are sequentially formed to an original pat- 
tern. 

[0093] The image forming unit 43 forms image data 
by using the font data included in the font set 44 on the 
basis of the font size of the text data formed in step S01 . 15 
At this time, the font size of the image data may be 
made equal to the font size of the text data, or the font 
size of the image data may be expanded or reduced 
with respect to the font size of the text data. 
[0094] In this example, the image forming unit 43 20 
forms the image file (e.g., bit map file) of translation 
results (words and idioms translated into German) with 
respect to the destination "Mizel". 
[0095] The image forming unit 43 is a function real- 
ized such that the CPU 1 6 executes a program. 25 

((Step S08» 

[0096] In step S08, the display control unit 42 dis- 
plays the edition screen of each destination on the dis- 30 
play device 26. More specifically, the display control unit 
42 displays an edition screen in the edition region 55. In 
the edition screen, the text file formed in step S01 and 
the text file formed in step S06 are laid out in accord- 
ance with predetermined format information. 35 
[0097] The format information includes information 
of a position (left-end position of the sentences) repre- 
senting a position at which the Japanese sentences are 
started, a line space of characters, and a width between 
lines (right-end position of the edition region 55). The 4c 
display control unit 42 displays the text of a translated 
word or idiom at a corresponding position of an imagi- 
nary sheet (text display sheet) on which the Japanese 
sentences are arranged in the edition region 55. 
[0098] More specifically, in the edition region 55, 41 
Japanese sentences are text-displayed at intervals 
each corresponding to one line. The words and idioms 
translated into a foreign language are text-displayed 
above the corresponding Japanese sentences such that 
the words and the idioms are adjusted to the left side. a 
[0099] Fig. 11 A is a diagram showing a display 
example of an edition screen corresponding to the des- 
tination "Bob", and Fig. 1 1B is a diagram showing a 
manner obtained after an edition operation is performed 
to the edition screen shown in Fig. 1 1 A. 5 
[0100] On th other hand, the image file is formed in 
step S07, the display control unit 42 displays an edition 
screen on which the t xtfiie formed in step S01 and the 



image file formed in step S07 are laid out in accordance 
with the predetermined format information on the edi- 
tion region 55. 

[0101] Fig. 12A is a diagram showing a display 
example of an edition screen corresponding to the des- 
tination "Mizel", and Fig. 12B is a diagram showing a 
manner obtained after an edition operation is performed 
to the edition screen shown in Fig. 12A. Fig. 13 is a dia- 
gram for explaining display control performed by the dis- 
play control unit 42. 

[0102] The format information includes information 
of a position (left-end position of the sentences) repre- 
senting a position at which the Japanese sentences are 
started, a line space of characters, and a width between 
lines (right-end position of the edition region 55). As 
shown in Fig. 13, the display control unit 42 pastes the 
image of a translated word or idiom at a corresponding 
position of an imaginary sheet (text display sheet) on 
which the Japanese sentences. 

[0103] Fig. 13 shows an example in which a docu- 
ment image "Gooden Tag." (file name: terml.bmp) is 
pasted above a Japanese sentence " konnichiwa' . 
According'to this method, Japanese sentences are text- 
displayed as an edition screen at intervals each corre- 
sponding to one line in the edition region 55 shown in 
Fig. 12A. The document image (portion enclosed by a 
broken line in Fig. 12A) of the translated word or idiom 
is displayed above the Japanese sentence correspond- 
ing to the word or idiom such that the word or the idiom 
is adjusted to the left side. 

[01 04] A user refers to the displayed translated sen- 
tence. If there is a mistranslation, the user operates the 
KBD 27 to correct the mistranslation. The display con- 
trol unit 42 displays a symbol • , ?????" above a word 
which cannot be translated. The user operates the KBD 
27 to input the equivalent of the word in place of the 
symbol "????? p (see Fig. 11B). 

[0105] Fig. 14 is a diagram for explaining an editing 
operation. As shown in Fig. 14, the user operates the 
KBD 27 to select a portion to be translated or corrected 
in the Japanese sentences displayed on the edition 
screen (edition region 55), and marks (inverts) the cor- 
responding portion. 

[0106] When the user presses the translation but- 
ton 56 by using the KBD 27 or the mouse 28, the display 
control unit 42 displays a translated word display win- 
dow 59 at a predetermined position of the edition region 
55 such that the translated word display window 59 
overlaps the text display sheet. The translated word dis- 
> play window 59 may be displayed at any position on the 
screen of the display device 26. 

[0107] On the other hand, the translation unit 48 
translates the marked portion into a foreign language 
(English in an example in Fig. 14) corresponding to a 
; destination, and gives th translation results to the dis- 
play control unit 42. The display control unit 42 text-dis- 
plays the translation results (translated words) on the 
translated word display window 59. At this time, when 
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the translation results are not proper, the user can cor- 
rect the translated words displayed on the translated 
word display window 59 by the operation of the KBD 27. 
[0108] When the translation results cannot be text- 
displayed, the translated words are image-displayed on 
the translated word display window 59. When the user is 
to correct the image-displayed translated words, the 
user designates the code of fonts for forming the trans- 
lated words. 

[0109] At this time, the corresponding font data is 
read from the font set 44, the image file of desired trans- 
lated words is formed by the image forming unit 43, and 
the image of the translated words is displayed on the 
translated word display window 59 by the display control 
unit 42. 

[0110] Thereafter, when the user fixes the trans- 
lated words displayed on the translated word display 
window 59 by the operation of the KBD 27 or the mouse 
28, the translated words are image-displayed above the 
marked Japanese sentence. In this manner, if trans- 
lated words or a translated sentence written in a foreign 
language is displayed as either one of a text and an 
image, the user can edit the translated words or the 
translated sentence. 

[0111] According to the method described above, 
translation results corresponding to the destination 
"Bob - and the destination "MizeP are corrected (see 
Fig. 11 B and Fig. 12B). Therefore, if the terminal device 
T1 cannot text-display German, words and idioms trans- 
lated into German can be displayed on the display 
device 26 and can be edited. 

[0112] The edition results (corrected contents) of 
the translation described above are reflected on the 
next translation by the learning function of the transla- 
tion unit 48. 

({Step S09» 

[0113] In step S09, the third determination unit 51 
checks whether the language in the transmission 
source can be text-displayed in the destination. More 
specifically, the third determination unit 51 refers to "dis- 
play in Japanese" in records corresponding to destina- 
tions included in the address list 52 (see Fig. 7). When 
"display in Japanese" is "OK", the third determination 
unit 51 determines that it is possible to perform text dis- 
play. When "display in Japanese" is "NG", the third 
determination unit 51 determines that it is impossible to 
perform text display. 

[0114] In this example, the third determination unit 
51 determines that it is impossible to perform text dis- 
play in Japanese in the destination "Bob" (terminal 
device T2), and determines that it is possible to perform 
text display in Japanese in the destination "Mizel" (ter- 
minal device T3). 

[0115] The third determination unit 51 is a function 
realized such that the CPU 16 executes a program. 



16 

({Step S10)> 

[0116] in step S10, the synthesizing unit 45 forms a 
synthesized image file obtained by synthesizing an orig- 

5 inal (Japanese sentence) and a translated sentence. 
Fig. 15 is a diagram for explaining a synthesizing proc- 
ess performed by the synthesizing unit 45. The synthe- 
sizing unit 45 pastes the edited image files at 
predetermined positions in accordance with predeter- 

10 mined format information. 

[0117] Subsequently, the synthesizing unit 45 con- 
verts the text file formed in step S01 into an image file 
(bitmap file), and pastes the image file at a predeter- 
mined position. As a pre-process of this synthesizing 

is process, it is determined in step S05 that the language 
in the destination can be text-displayed. When the 
image file of the translation results is not formed, the 
image file of the translation results is formed. In this 
manner, a synthesized image file obtained by synthesiz- 

20 ing an original and a translated sentence is formed. In 
this example, a synthesized image file corresponding to 
the destination "Bob" is formed. 

[0118] The synthesizing unit 45 is a function real- 
ized such that the CPU 1 6 executes a program. 

25 

{{Step S11)> 

[0119] In step S11, the electronic mail formed in 
steps S01 to S10 is transmitted to the destination. More 
30 specifically, the contents of the electronic mail subjected 
to the process in step S08 or step S10 are displayed in 
the edition region 55. 

[0120] At this time, when a user presses the trans- 
mission button 58 by the operation of the KBD 27 or the 
35 mouse 28, the transmission unit 46 transmits the 
formed electronic mail in accordance with the mail 
address. 

[0121] In this case, when the sentence to be trans- 
mitted as an electronic mail is only a text file of a Japa- 

40 nese sentence (original), or when the Japanese 
sentence and the translated portion of the Japanese 
sentence are formed as text files, the transmission unit 
46 transmits the contents of these text files as the body 
of the electronic mail in accordance with the mail 

45 address of the destination. 

[0122] In contrast, when a Japanese sentence is 
formed as a text file, and when a translated portion is 
formed as an image file, a file corresponding to the dis- 
play contents obtained upon completion of step S08. 

so The transmission unit 46 transmits the formed file as an 
attached file. 

[0123] In this example, in an electronic mail to the 
destination "MizeP', the Japanese sentence is formed as 
a text file, and the translated portion in German is 
55 formed as an image file. For this reason, a file obtained 
by combining the text file and the image file is formed, 
and this file is transmitted as an attached file to the des- 
tination "Mizel" (terminal device T3) (see Fig. 16). 
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[0124] In contract to this, the Japanese sentence 
and the translated portion are formed as image files, the 
display contents (synthesized image file) obtained upon 
completion of step S10 are transmitted as an attached 
file to the destination. The transmission unit 46 is a func- 
tion obtained such that the CPU 16 and/or the commu- 
nication control device 23 is operated in accordance 
with a predetermined control program. 
[0125] In this example, the Japanese sentence and 
the translated portion of an electronic mail to the desti- 
nation "Bob" are formed as image files. For this reason, 
the transmission unit 46 transmits the synthesized 
image file formed in step S1 0 as an attached file to the 
destination "Bob" (terminal device T2) (see Fig. 16). 
[0126] At this time, the electronic mail may be 
directly transmitted to each destination, or the electronic 
mail may be uploaded from the terminal device T1 to a 
server in a network NW, and the terminal devices T2 
and T3 may download the electronic mail transmitted 
from the terminal device T1 from the server as needed. 
[0127] Thereafter, when the destination "Bob" 
starts a mailer in the terminal device T2 and opens the 
attached file in accordance with the sentence "Please 
read attached file." in the electronic mail transmitted in 
step S1 1, the image constituted by the Japanese sen- 
tence and the translated portion is displayed on the dis- 
play device of the terminal device T2. 
[0128] When the destination "Mizel" starts a mailer 
in the terminal device T3 and opens the electronic mail 
transmitted in step S11, the contents of the electronic 
mail in which the Japanese sentence is text-displayed 
and the translated portion is image-displayed are dis- 
played on the display device of the terminal device T3. 
[0129] In the description of steps S01 to S11, the 
process related to the destination "Bob- and the proc- 
ess related to the destination "Mizel" are performed in 
the steps in parallel. However, in fact, the processes in 
steps S01 to S1 1 are performed every destination. Note 
that the processes may be executed in parallel. 
[0130] The above embodiment describes the exam- 
ple in which the present invention is applied to a termi- 
nal (terminal device T1) on a transmission side. 
However, the present invention is not limited to the ter- 
minal on the transmission side, and the present inven- 
tion can also applied to another device in a network to 
which a terminal on a transmission side belongs or a 
device in a network to which a terminal on a reception 
side belongs. 

[0131] As an example of the device, a mail server or 
a proxy server arranged on a network to which a termi- 
nal on a transmission side or a terminal on a reception 
side belongs, or the terminal on the reception side is 
known. 

[0132] Since the hardware configuration of a mail 
server for executing the processes in an embodiment of 
the present invention may be the same as that of the ter- 
minal device T1 described by using Fig. 2, a description 
thereof will be omitted. However, when operations 



embodying the present invention are executed by a mail 
server, several constituent elements such as the display 
26, the keyboard 27, and the mouse 28 are not neces- 
sary. 

5 

(Process in Mail Server) 

[0133] The mail server functions as an electronic 
mail apparatus which performs the following operations 
10 such that the CPU executes a program. Fig. 1 7 is a flow 
chart showing operations in the mail server. 
[0134] First, the mail server receives an electronic 
mail in which a sentence (body) is written in the mother 
tongue of a user of the terminal device T1 from the ter- 
15 minal device T1 (step S21). More specifically, the CPU 
executes a reception program for an electronic mail 
when it receives the electronic mail transmitted from the 
terminal device T1 . 

[0135] It is checked whether the language identical 
20 to that in the terminal device T1 serving as the transmis- 
sion source of the electronic mail is used in the destina- 
tion or not (step S22). At this time, if it is determined the 
languages are identical to each other, the operation 
shifts to step S28. If it is determined that the languages 
25 are different from each other, the operation shifts to step 
S23. 

[0136] The above determination is performed by 
the CPU such that prepared determination data is used. 
However, the CPU may find a language used in the des- 
30 tination on the basis of country information included in 
the destination address of the electronic mail received 
in step S21 to automatically check whether the lan- 
guage used in the transmission source is identical to the 
language used in the destination. 
35 [0137] In step S23, the language (mother tongue) in 
the destination is specified, and the contents (body) of 
the electronic mail received in step S21 are translated in 
the specified language. However, the CPU may find a 
language on the basis of the destination address. 
40 [0138] In step S24, after the CPU forms the text file 
of results (translation results) of the translation process 
in step S23, the operation shifts to step S25. 
[0139] In step S25, it is checked whether the char- 
acters written in the language (Japanese) of the trans- 
45 mission source can be text-displayed in the destination 
or not. 

[0140] In step S26, the CPU forms image data cor- 
responding to the text data of the body (sentence 
formed by characters written in the language in the 
so transmission source) of the electronic mail received in 
step S21 and forms image data corresponding to the 
text data of the translation results on the basis of the text 
file of the translation results formed in step S24 to form 
a synthesized imag file obtained by synthesizing these 
55 image data. 

[0141] In step S27, the CPU forms text data 
obtained by synthesizing the body (sentence formed by 
characters writt n in the language in the transmission 
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source) of the electronic mail received in step S21 , the 
text data of the translation results, and the text data of 
the body of the electronic mail to form a text file. 
[0142] In step S28, the CPU transmits the elec- 
tronic mails formed in the above steps to a destination 
indicated by the header information of the electronic 
mail received in step S21 . 

[0143] The transmitted electronic mail is as follows. 

(1) If "YES" is determined in step S22, the sentence 
(the body of the received mail) formed in the lan- 
guage in the transmission source is directly used. 

(2) If "NO" is determined in step S22, or if "YES" is 
determined in step S25, a sentence obtained by 
synthesizing the body of the received electronic 
mail and the contents of the text file of the transla- 
tion results is used as the body of the electronic 
mail. 

(3) If "NO" is determined in step S22 and step S25, 
a synthesized image file obtained by synthesizing 
the image file formed from the body of the received 
electronic mail and the image file of the translation 
results is used as an attached file. 

(Detailed Operation of Mail Server) 

[0144] Fig. 18 is a functional block diagram of the 
mail server. In the mail server, the CPU executes a pro- 
gram to perform the operations in steps S21 to S28 
described above. In this manner, the mail server func- 
tions as a device comprising a reception unit 61, a text 
forming unit 62, an image forming unit 63, a font set 
(data set of fonts) 64, a synthesizing unit 65, a transmis- 
sion unit 66, a first determination unit 67, a translation 
unit 68, a dictionary 69, and a third determination unit 
70 as shown in Fig. 1 8. 

[0145] The details of steps S21 to S28 shown in 
Fig. 17 will be described below by using Fig. 18. 

((Step S21» 

[0146] In step S21, an electronic mail, transmitted 
from the terminal device (transmission source terminal) 
T1 of the transmission source, in which a sentence 
(body) is written in a mother tongue by a user of the 
transmission source terminal T1 is received by the 
reception unit 61 . 

((Step S22» 

[0147] In step S22, the first determination unit 67 
checks whether the language in the destination indi- 
cated by the header information of the electronic mail 
received in step S21 is identical to the language in the 
transmission source or not. More specifically, the first 
determination unit 67 holds a plurality of records includ- 
ing at least information representing that Japanese is 
used as the language in the transmission source and 



pieces of information corresponding to the items "mail 
address", "mother tongue", and "display in Japanese" of 
the address list 52 shown in Fig. 7 as determination 
data in advance. The first determination unit 67 checks 

5 whether the "mother tongue" in the records correspond- 
ing to the destination mail address is Japanese or not. 
[0148] In this manner, the first determination unit 67 
automatically performs determination by using the infor- 
mation of the language (Japanese) in the transmission 

70 source prepared as determination data and the informa- 
tion of the "mother tongue" corresponding to the desti- 
nation mail address. 

[0149] When the language in the transmission 
source is different from the language in the destination 

15 (when the language in the destination is not Japanese), 
the first determination unit 67 notifies the translation unit 
68 that these languages are different from each other. In 
contrast to this, when the language in the destination is 
Japanese, the first determination unit 67 does not notify 

20 the translation unit 68. The first determination unit 67 is 
a function realized such that the CPU executes a pro- 
gram. 



25 



((Step S23» 



[0150] In step S23, the translation unit 68 is started 
by receiving a notification from the first determination 
unit 67, and performs the following processes. More 
specifically, the translation unit 68 refers to the "mother 

30 tongue" in the records corresponding to the destination 
in which it is determined that the language is not Japa- 
nese. In this manner, the translation unit 68 specifies a 
language to be translated. The first determination unit 
67 may specify a language to be translated to notify the 

35 translation unit 68. 

[0151] Subsequently, the translation unit 68 refers 
to the dictionary (basic dictionary) 69. In this manner, 
the translation unit 68 transmits the body (Japanese 
sentence) of the electronic mail received in step S21 

40 into the specified language. In this case, the dictionary 
69 is constituted by dictionaries for n languages such as 
a Japanese-English dictionary or a Japanese-German 
dictionary for translating Japanese into a foreign lan- 
guage. 

45 [0152] When the translation unit 68 translates the 
Japanese sentence, the translation unit 68 extracts 
words or idioms of one sentence to translate the 
extracted words or idioms into the corresponding for- 
eign language. When the words or the idioms ar 

so extracted, for example, decomposition of parts of 
speech is performed by explosion morpheme analysis, 
and a translation for a specific part of speech (e.g., noun 
or verb) is performed. 

[01 53] The translation unit 68 does not output trans- 
55 lation results to a word which is not described in the dic- 
tionary 69. The translation unit 68 may translate not only 
the words and idioms in one sentence but also the 
whole sentence. 
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[0154] The translation unit 68 is a function realized 
such that the CPU executes a translation program, and 
any conventional translation program can also be used 
as the translation program. 

{(Step S24» 

[0155] In step S24, the text forming unit 62 receives 
translation results from the translation unit 68. The text 
forming unit 62 forms text data corresponding to a 
words and idioms included in the translation results and 
translated into a foreign language to hold the image 
data as a text file. 

{{Step S25» 

[0156] In step S25, the third determination unit 70 
checks whether the language (the body of the received 
electronic mail) in the transmission source can be text- 
displayed in the destination or not More specifically, the 
third determination unit 70 refers to the "display in Jap- 
anese" in the records corresponding to the destination 
and included in the held determination data. At this 
time, if the display in Japanese is "OK", the third deter- 
mination unit 70 determines that it is possible to perform 
text display. If the display in Japanese is "NG", the third 
determination unit 70 determines that it is impossible to 
perform text display. The third determination unit 70 is a 
function realized such that the CPU executes a pro- 
gram. 

{{Step S26» 

[0157] In step S26, the image forming unit 63 forms 
image data corresponding to the text data of the body of 
the received electronic mail and the translation results 
(translated portions) by using the text data. More specif- 
ically, the image forming unit 63 forms image data on 
the basis of the text data of the body of the received 
electronic mail by using a Japanese font data included 
in the font set 64. 

[0158] The image forming unit 63 forms image data 
on the basis of the text data of the translation results by 
using the font data of the mother tongue included in the 
font set 64 and used in the destination of the electronic 
mail. A font size for forming the image data is set in 
advance. 

[0159] The synthesizing unit 65 synthesizes the 
image data of the formed Japanese sentence and the 
translated portion according to predetermined format 
information (line space, line width, and the like) such 
that the Japanese sentence and the translated portion 
are paired, thereby forming an image file. 
[0160] The image forming unit 63 and the synthe- 
sizing unit 65 are functions realized such that the CPU 
executes programs. 



{{Step S27» 

[0161] In step S27, the synthesizing unit 65 forms a 
text file which is synthesized by using the text data of 
5 the body of the received electronic mail and the text file 
of the translation results formed by the text forming unit 
62 such that the Japanese sentence and the text data of 
the translated portion are paired. 

w {{Step S28» 

[0162] In step S28, the transmission unit 66 trans- 
mits the electronic mails constituted by the contents 
formed by the above processes to a destination. More 
15 specifically, when the image file constituted by the Jap- 
anese sentence and the translated portion is formed in 
step S26, an electronic mail in which the image file is 
used as an attached file and a sentence (e.g., "Please 
read attached file.") attached with a file is used as a 
20 body is transmitted to the destination. 

[0163] When a text file constituted by a Japanese 
sentence and the translated portion thereof is formed in 
step S27, an electronic mail in which the text data of the 
text file is used as a body is transmitted to the destina- 
25 tion. As described above, an embodiment of the present 
invention can also be used in a device other than a 
transmission source terminal (terminal device T1). 
[0164] Although the device connected to a network 
to which a transmission source terminal belongs has 
30 been described above, an embodiment of the present 
invention can be used even in a device connected to a 
network to which the terminal device of a destination 
(destination terminal) belongs. In this case, the item 
"mail address" is set to be a transmission source mail 
35 address, the item "mother tongue" is set to be the 
mother tongue of a user of the transmission source mail 
address, and the item "display in Japanese" is set to be 
information representing whether the mother tongue of 
the transmission source can be displayed or not. 
40 [0165] Therefore, when the item "mother tongue" 
corresponding to the transmission source mail address 
of the received electronic mail is different from the 
mother tongue used in the destination terminal, the 
body of the received electronic mail is translated. When 
45 it is represented that the mother tongue in the transmis- 
sion source can be displayed, a test file constituted by 
the body of the received electronic mail and the trans- 
lated portion of the body is formed; otherwise, an image 
file constituted by the body of the electronic mail and the 
so translated portion of the body is formed. After the same 
synthesizing process as described above is performed, 
the image file is transmitted to the destination terminal 
by a predetermined method. 

[0166] In this manner, an embodiment of the 
55 present invention can be applied to either one of the 
device on the transmission source side and the device 
on the destination side. In contrast to this, in a device 
other then the terminal device of the transmission 
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source, the translated portion of the formed original 
(Japanese sentence) cannot be edited. Therefore, 
when an embodiment of the present invention is applied 
to the terminal device of the transmission source, a 
translation having the highest precision can be per- 
formed, and a receiver of the electronic mail easily 
understands the contents of the electronic mail. 
[0167] In addition, in an embodiment of the present 
invention, there may be a case in which the image data 
of a sentence (original) input as the contents of an elec- 
tronic mail and the text data of the translated sentence 
are transmitted to the destination of the electronic mail 
and a case in which both the original and the translated 
sentence of the original are translated as image data. 
[0168] These cases have advantages, respectively. 
In the former case, since the translated sentence is text 
data, the capacity of data to be transmitted to the desti- 
nation of the electronic mail can be suppressed. In the 
later case, since both the data are image data, the ter- 
minal device (destination terminal) on the reception side 
does not require a function in which, as in the former 
case, the received text data and the image data are syn- 
thesized with each other to be displayed and which is 
not held by a general electronic mail apparatus (elec- 
tronic mail software), and an electronic mail apparatus 
used as present can be directly used. 

{Operation of Embodiment) 

[0169] According to the electronic mail apparatus 
according to the embodiment described above, when 
characters (hiragana, katakana, and kanji) for writing 
Japanese cannot be text-displayed in a destination, the 
image of a sentence constituting the contents of an 
electronic mail formed in Japanese is formed, and the 
image is transmitted to the destination. In this manner, 
in the destination, a sentence formed in Japanese can 
be displayed on a display device. For this reason, a 
receiver of an electronic mail in the destination can read 
an electronic mail formed in Japanese. The embodi- 
ment has been described with respect to Japanese. 
However, the same operation as described above when 
the embodiment is made for another language. 
[0170] When words or idioms are translated, the 
translation results cannot be text-displayed in a trans- 
mission source. In this case, an image file correspond- 
ing to the translation results is formed, and a document 
image based on the image file is displayed on the dis- 
play device 26. For this reason, a person who forms an 
electronic mail refers to the translation results, and can 
edit (correct) the translation results. 
[0171] In the embodiment described above, format 
information is held to lay out an image in a text sen- 
tence. However, in place of this, a configuration may be 
formed such that a line space and the like are deter- 
mined in accordance with the font size of the text. 
[0172] In this case, information (the position of the 
left end, a line space, and a line width (the position of 



the right end)) required to lay out an image can be cal- 
culated such that position information of Japanese sen- 
tences, line space information, and the position of the 
right end of the longest sentence in the sentences are 
5 calculated on the basis of the font size of the Japanese 
sentences. 

[0173] In this embodiment, although a Japanese 
sentence and the translated portion thereof are dis- 
played on the same screen, the Japanese sentence and 

w the translated portion thereof may be displayed on dif- 
ferent screens, respectively. At this time, the two 
screens may be displayed to be arrayed, and one of the 
two screens may be displayed by screen switching. 
[0174] In addition to the embodiment described 

is above, a device and method for performing the following 
processes, and a recording medium on which a pro- 
gram is recorded for causing a computer to execute the 
processes, also embody the present invention. 
[0175] The process contents are as follows. That is, 

20 when, by using a sentence input as the contents of an 
electronic mail, image data corresponding to the sen- 
tence is formed, and when the sentence is expressed in 
a language different from a language used in a destina- 
tion of the mail, the sentence is translated into the lan- 

25 guage used in the destination. The text data of the 
translation results is editably displayed on a screen, and 
the formed image data and the translation results are 
transmitted to the destination of the electronic mail. 
[0176] In this manner, a correcting operation of the 

30 translation results corresponding to the input sentence 
can be performed, and the appropriately corrected 
translated sentence and the image data of the sentence 
serving as the original can be transmitted to the destina- 
tion. 

35 [0177] The translated text data may be displayed 
such that the text data corresponds to the sentence. In 
this manner, the correspondence between the input 
sentence and the translated sentence thereof can be 
easily understood, and the correcting operation can be 

40 easily performed. 

[0178] In addition, the other process contents are 
as follows. That is, when, by using a sentence input as 
the contents of an electronic mail, image data corre- 
sponding to the sentence is formed, and when the sen- 

45 tence is expressed by characters written in a language 
different from a language used in a destination of the 
electronic mail, the sentence is translated into the lan- 
guage used in the destination. Translated image data 
corresponding to the translation results is formed by the 

so characters written in the language used in the destina- 
tion, the translated image data is editably displayed to 
correspond to the sentence, and the formed image data 
and the translation results are transmitted to the desti- 
nation of the electronic mail. 

55 [0179] In this manner, characters written in the lan- 
guage which cannot be text-displayed in the transmis- 
sion source and is used in the destination can be 
displayed, and a correcting operation for the translation 
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results corresponding to the input sentence can be per- 
formed. Therefore, the appropriately corrected trans- 
lated sentence and the image data of the sentence 
serving as the original can be transmitted to the destina- 
tion. 5 
[0180] The above description has referred to a com- 
puter program stored on a computer-readable (or 
device- readable) medium. It will be appreciated that 
such a computer program (or operating program) can 
be used to operate any suitable electronic mail device w 
(such as a personal computer, a work station, a mobile 
computer or a mobile phone). It will also be appreciated 
that an operating program embodying the present 
invention need not be stored on a device-readable 
medium and could, for example, be embodied in a sig- is 
nal such as a downloadable data signal provided from 
an Internet website. The appended claims are to be 
interpreted as covering an operating program by itself, 
or as a record on a carrier, or as a signal, or in any other 
form. 20 

Claims 

1. An electronic mail apparatus comprising: 



25 



30 



35 



an image data forming unit using a sentence 
input as the contents of an electronic mail to 
form image data corresponding to the sen- 
tence; and 

a transmission unit transmitting the formed 
image data to a destination of the electronic 
mail. 

An electronic mail apparatus according to claim 1, 
further comprising a determination unit checking 
whether the sentence should be transmitted as text 
data or image data, 

wherein the image data forming unit, when the 
determination unit determines that the sentence 
should be transmitted as image data, forming 
image data corresponding to the sentence by using 
the sentence. 



An electronic mail apparatus according to claim 2, 
wherein the determination unit determines that the 45 
sentence should be transmitted as image data 
when the text data of the sentence is not correctly 
displayed in the destination of the electronic mail. 

An electronic mail apparatus according to claim 1,2 so 
or 3, further comprising a display control unit using 
text data of the sentence to display the sentence on 
a screen, 

wherein the image data forming unit forms image 
data corresponding to a sentence written in a for- 55 
mat displayed on the screen by the display control 
unit. 



5. An electronic mail apparatus according to any pre- 
ceding claim, further comprising a translation unit 
translating the sentence into a language used in the 
destination when the sentence is expressed by 
characters written in a language different from the 
language used in the destination of the electronic 
mail, 

wherein the transmission unit transmits translation 
results obtained by the transmission unit to the des- 
tination of the electronic mail. 

6. Art electronic mail apparatus according to claim 5, 
wherein the image forming unit forms translated 
image data of the translation results expressed by 
characters written in the language used in the des- 
tination when the display control unit cannot text- 
display characters written in the language used in 
the destination of the electronic mail, and 

the display control unit displays a screen on 
which the translated image data is edited. 

7. An electronic mail apparatus according to claim 5, 
wherein the image data forming unit forms the 
translated image data of the translation results 
expressed by characters written in the language 
used in the destination when the display control unit 
cannot text-display the characters written in the lan- 
guage used in the destination of the electronic mail, 
and 

the electronic mail apparatus further comprises 
a synthesizing unit forming synthesized image 
data obtained by combining image data formed 
by the image data forming unit and correspond- 
ing to the input sentence with the translated 
image data, and 

the transmission unit transmits the synthesized 
image data to the destination of the electronic 
mail. 

8. An electronic mail apparatus according to an pre- 
ceding claim, wherein the transmission unit trans- 
mits the image data as an attached file to the 
destination of the electronic mail. 

9. An electronic mail transmission method comprising: 

the input step of inputting a sentence serving 

as the contents of an electronic mail; 

the image data forming step of forming image 

data corresponding to the input sentence by 

using the input sentence; and 

the transmission step of transmitting the 

formed image data to the destination of the 

electronic mail. 

10. An electronic mail transmission method according 
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to claim 9, further comprising 

the determination step of checking whether the 
input sentence should be transmitted as text 
data or image data, 5 
wherein, when it is determined by the determi- 
nation that the sentence should be transmitted 
as image data, in the transmission step, the 
image data is transmitted to the destination of 
the electronic mail. io 

11. An electronic mail transmission method according 
to claim 1 0, 

wherein, in the determination step, when it is deter- 
mined that the text data of the input sentence can- 15 
not be correctly displayed in the destination of the 
electronic mail, it is determined that the image data 
should be transmitted. 

12. An electronic mail transmission method according 20 
to claim 9, 10 or 1 1, further comprising the display 
control step of using text data of the sentence to 
display the sentence on a screen, 

wherein, in the image data forming step, image 
data corresponding to a sentence written in a for- 25 
mat displayed on the screen is formed. 

13. An electronic mail transmission method according 
to any one of claims 9 to 12, further comprising the 
translation step of translating the sentence into a 30 
language used in the destination when the input 
sentence is expressed by characters written in a 
language different from the language used in the 
destination of the electronic mail, 

wherein, in the transmission step, translation 35 
results are transmitted to the destination of the 
electronic mail. 

14. An electronic mail transmission method according 

to claim 1 3, 40 
wherein the image forming step forms translated 
image data of the translation results expressed by 
characters written in the language used in the des- 
tination when the characters written in the language 
used in the destination of the electronic mail cannot 45 
be text-displayed, and 

the electronic mail transmission method further 
comprises the image edition step of editably 
displaying the translated image data on a so 
screen. 

15. An electronic mail transmission method according 
to claim 13, 

wherein the image data forming step forms trans- 55 
lated image data of the translation results 
expressed by characters written in the language 
used in the destination when the characters written 



in the language used in the destination of the elec- 
tronic mail cannot be text- displayed, and 

the electronic mail transmission method further 
comprises the synthesizing step of forming 
synthesized image data obtained by combining 
the formed image data corresponding to the 
input sentence with the translated image data, 
and 

in the transmission step, the synthesized image 
data is transmitted to the destination of the 
electronic mail. 

16. An electronic mail transmission method according 
to any one of claims 9 to 15, wherein the image 
data is transmitted as an attached file to the desti- 
nation of the electronic mail. 

17. A computer-readable recording medium which 
records a program for causing a computer to exe- 
cute: 

the input step of inputting a sentence serving 

as the contents of an electronic mail; 

the image data forming step of forming image 

data corresponding to the input sentence by 

using the input sentence; and 

the transmission step of transmitting the 

formed image data to the destination of the 

electronic mail. 

18. A computer-readable recording medium according 
to claim 17, which records the program further 
including 

the determination step of checking whether trie 
input sentence should be transmitted as text 
data or image data, 

wherein, when it is determined by the determi- 
nation that the sentence should be transmitted 
as image data, in the transmission step, the 
image data is transmitted to the destination of 
the electronic mail. 

19. A computer-readable recording medium according 
to claim 18, which records the program 
wherein, in the determination step, when it is deter- 
mined that the text data of the input sentence can- 
not be correctly displayed in the destination of the 
electronic mail, it is determined that the image data 
should be transmitted. 

20. A computer-readable recording medium according 
to claim 17, 18 or 19, which records the program 
further including the display control step of using 
text data of the sentence to display the sentence on 
a screen, 

wherein, in the image data forming step, image 
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data corresponding to a sentence written in a for- 
mat displayed on the screen is formed. 

21. A computer-readable recording medium according 
to any one of claims 17 to 20, which records the 5 
program 

further comprising the translation step of trans- 
lating the sentence into a language used in the 
destination when the input sentence is 10 
expressed by characters written in a language 
different from the language used in the destina- 
tion of the electronic mail, 
wherein, in the transmission step, translation 
results are transmitted to the destination of the is 
electronic mail. 



inputting a sentence serving as the contents of 
an electronic mail; 

forming image data corresponding to the input 
sentence by using the sentence; and 
transmitting the formed image data to the des- 
tination of the electronic mail. 

26. An operating program as claimed in claim 25, car- 
ried in a carrier medium. 

27. An operating program as claimed in claim 26, 
wherein the carrier medium is a transmission 
medium. 

28. An operating program as claimed in claim 26, 
wherein the carrier medium is a storage medium. 



22. A computer-readable recording medium according 
to claim 21, which records the program 

wherein the image forming step forms translated 20 
image data of the translation results expressed by 
characters written in the language used in the des- 
tination when the characters written in the language 
used in the destination of the electronic mail cannot 
be text-displayed, and 25 

the program further comprises the image edi- 
tion step of editably displaying the translated 
image data on a screen. 

30 

23. A computer-readable recording medium according 
to claim 21, which records the program 

wherein the image data forming step forms trans- 
lated image data of the translation results 
expressed by characters written in the language 35 
used in the destination when the characters written 
in the language used in the destination of the elec- 
tronic mail cannot be text-displayed, and 



the program further comprises the synthesizing 40 
step of forming synthesized image data 
obtained by combining the formed image data 
corresponding to the input sentence with the 
translated image data, and 

in the transmission step, the synthesized image 45 
data is transmitted to the destination of the 
electronic mail. 



24. A computer-readable recording medium according 
to any one of claims 17 to 23, which records the so 
program wherein the image data is transmitted as 
an attached file to the destination of the electronic 
mail. 



25. An operating program which, when run on elec- 55 
tronic mail apparatus, causes the apparatus to 
carry out the steps of: 
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